Traditionally, testing devices and device drivers has been a highly manual process. These manual tests are time consuming and expensive to run in a consistent way. Even when these tests are automated, the resulting automation code has several problems. The existing automated tests for devices and device drivers require a high level of knowledge about specific hardware devices and technologies in order to write code to control and test these devices. Further, the automated tests include monolithic code that cannot be easily maintained over time and are difficult to modify or re-use, and do not lend themselves to componentization (e.g., being a part of another test). Finally, the automated tests are difficult to code (e.g., written in C or C++) and cannot generally be interrupted and restarted.
Additionally, the conventional test architecture has organizational efficiency problems. First, it is difficult to distribute test automation efforts. Additionally, tests do not generally record metrics in any consistent way (e.g., success or failure, number of executions, devices tested etc.)